Smoothing Multivariate Performance Measures

A Support Vector Method for multivariate performance measures was recently introduced by Joachims (2005). The underlying optimization problem is currently solved using cutting plane methods such as SVM-Perf and BMRM. One can show that these algorithms converge to an accurate solution in O ( 1 λ ) iterations, where λ is the trade-off parameter between the regularizer and the loss function. We present a smoothing strategy for multivariate performance scores, in particular precision/recall break-even point and ROCArea. When combined with Nesterov’s accelerated gradient algorithm our smoothing strategy yields an optimization algorithm which converges to an accurate solution in O∗ ( min { 1 , 1 √ λ }) iterations. Furthermore, the cost per iteration of our scheme is the same as that of SVM-Perf and BMRM. Empirical evaluation on a number of publicly available datasets shows that our method converges significantly faster than cutting plane methods without sacrificing generalization ability. 1 Background and Introduction Different kinds of applications served by machine learning algorithms have varied and specific measures to judge the performance of the algorithms. In this paper we focus on efficient algorithms for directly optimizing multivariate performance measures such as precision/recall break-even point (PRBEP) and area under the Receiver Operating Characteristic curve (ROCArea). Given a training set with n examples X := {(xi, yi)}i=1 where xi ∈ R and yi ∈ {+1,−1}, Joachims (2005) proposed an elegant formulation for this problem which minimizes the following regularized risk: min w J(w) = λ 2 ‖w‖ +Remp(w). (1) Here 12 ‖w‖ 2 is the regularizer, λ > 0 is a trade-off parameter and the empirical risk Remp for contingency table based multivariate performance measures is Remp(w)= max z∈{−1,1} [